Reliability and validity of the AGREE instrument used by physical therapists in assessment of clinical practice guidelines

BACKGROUND: The AGREE instrument has been validated for evaluating Clinical Practice Guidelines (CPG) pertaining to medical care. This study evaluated the reliability and validity of physical therapists using the AGREE to assess quality of CPGs relevant to physical therapy practice. METHODS: A total of 69 physical therapists participated and were classified as generalists, specialist or researchers. Pairs of appraisers within each category evaluated independently, a set of 6 CPG selected at random from a pool of 55 CPGs. RESULTS: Reliability between pairs of appraisers indicated low to high reliability depending on the domain and number of appraisers (0.17–0.81 for single appraiser; 0.30–0.96 when score averaged across a pair of appraisers). The highest reliability was achieved for Rigour of Development, which exceeded ICC> 0.79, if scores from pairs of appraisers were pooled. Adding more than 3 appraisers did not consistently improve reliability. Appraiser type did not determine reliability scores. End-users, including study participants and a separate sample of 102 physical therapy students, found the AGREE useful to guide critical appraisal. The construct validity of the AGREE was supported in that expected differences on Rigour of Development domains were observed between expert panels versus those with no/uncertain expertise (differences of 10–21% p = 0.09–0.001). Factor analysis with varimax rotation, produced a 4-factor solution that was similar, although not in exact agreement with the AGREE Domains. Validity was also supported by the correlation observed (Kendall-tao = 0.69) between Overall Assessment and the Rigour of Development domain. CONCLUSION: These findings suggest that the AGREE instrument is reliable and valid when used by physiotherapists to assess the quality of CPG pertaining to physical therapy health services

Inflammation in the male genital tract: implications for HIV acquisition and transmission

Elevated plasma levels of pro-inflammatory mediators such as TNFα, IL-1β, IL-6 and IL-8, MIP-1α, MIP-1β and RANTES have been demonstrated in HIV-infected individuals and HIV induces higher levels of pro-inflammatory cytokines in the female genital tract. We characterized levels of inflammation in semen, to gain an understanding of factors influencing transmission and acquisition in the male genital tract. Our hypothesis was that infected men would exhibit higher levels of inflammation in semen than uninfected men

Heart failure guidelines and prescribing in primary care across Europe

BACKGROUND: Major international differences in heart failure treatment have been repeatedly described, but the reasons for these differences remain unclear. National guideline recommendations might be a relevant factor. This study, therefore, explored variation of heart failure guideline recommendations in Europe. METHODS: Treatment recommendations of 14 national guidelines published after 1994 were analyzed in relation to the heart failure treatment guideline of the European Society of Cardiology. To test potential relations between recommendations and prescribing, national prescribing patterns as obtained by a European study in primary care (IMPROVEMENT-HF) were related to selected recommendations in those countries. RESULTS: Besides the 14 national guidelines used by primary care physicians in the countries contacted, the European guideline was used in four countries, and separate guidelines for specialists and primary care were available in another four countries. Two countries indicated that no guideline was used up to 2000. Comprehensiveness of the guidelines varied with respect to length, literature included and evidence ratings. Relevant differences in treatment recommendations were seen only in drug classes where evidence had changed recently (β-blockers and spironolactone). The relation between recommendation and prescribing for selected recommendations was inconsistent among countries. CONCLUSION: Differences in guideline recommendations are not sufficient to explain variation of prescribing among countries, thus other factors must be considered

Change of initial oral antidiabetic therapy in type 2 diabetic patients

Objective To explore the 'real-life' therapy of type 2 diabetes mellitus with oral antidiabetic drugs (OADs). Methods From the PHARMO Record Linkage System comprising linked drug dispensing and clinical laboratory data from approximately 2.5 million individuals in the Netherlands, among others, new users of OADs were identified in the period 1999-2004. New users, aged 30 years and older, without insulin use before cohort entry date and with at least one year follow-up were included. We determined per initial therapy patient characteristics and first therapy change. Results Overall 35,514 patients were included. Metformin and sulfonylureas (SU) were the most frequent initial therapy. Patients on thiazolidinedione (TZD) monotherapy had lower percentages baseline HbA1c ≥ 7% compared to patients on metformin and SU. The proportion of patients still on initial therapy after one year ranged from 46% (TZDs) to around 60% (SU). Among patients starting on monotherapy, add-on (15-20%) and discontinuation (16-25%) of therapy occurred most frequently. In patients starting on combination therapy, a switch occurred in 30% of the patients. Conclusion In more than 40% of the patients a change in initial OAD-therapy is already observed in the first year of therapy. Maintaining patients on initial therapy remains a challenge

Validation and application of an ensemble Kalman filter in the Selat Pauh of Singapore

The effectiveness of an ensemble Kalman filter (EnKF) is assessed in the Selat Pauh of Singapore using observing system simulation experiment. Perfect model experiments are first considered. The perfect model experiments examine the EnKF in reducing the initial perturbations with no further errors than those in the initial conditions. Current velocity at 15 observational sites from the true ocean is assimilated every hour into the false ocean. While EnKF reduces the initial velocity error during the first few hours, it fails after one tidal cycle (approximately 12 h) due to the rapid convergence of the ensemble members. Successively, errors are introduced in the surface wind forcing. A random perturbation ε [epsilon] is applied independently to each ensemble member to maintain the ensemble spread. The assimilation results showed that the success of EnKF depends critically on the presence of ε [epsilon], yet it is not sensitive to the magnitude of ε [epsilon], at least in the range of weak to moderate perturbations. Although all experiments were made with EnKF only, the results could be applicable in general to all other ensemble-based data assimilation methods.United States. Office of Naval ResearchSingapore. National Research FoundationSingapore-MIT Alliance for Research and Technology CenterSingapore-MIT Alliance. Center for Environmental Sensing and Monitorin

Improving the use of research evidence in guideline development: 13. Applicability, transferability and adaptation

BACKGROUND: The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the thirteenth of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this. OBJECTIVES: We reviewed the literature on applicability, transferability, and adaptation of guidelines. METHODS: We searched five databases for existing systematic reviews and relevant primary methodological research. We reviewed the titles of all citations and retrieved abstracts and full text articles if the citations appeared relevant to the topic. We checked the reference lists of articles relevant to the questions and used snowballing as a technique to obtain additional information. We used the definition "coming from, concerning or belonging to at least two or all nations" for the term international. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments. KEY QUESTIONS AND ANSWERS: We did not identify systematic reviews addressing the key questions. We found individual studies and projects published in the peer reviewed literature and on the Internet. Should WHO develop international recommendations? • Resources for developing high quality recommendations are limited. Internationally developed recommendations can facilitate access to and pooling of resources, reduce unnecessary duplication, and involve international scientists. • Priority should be given to international health problems and problems that are important in low and middle-income countries, where these advantages are likely to be greatest. • Factors that influence the transferability of recommendations across different settings should be considered systematically and flagged, including modifying factors, important variation in needs, values, costs and the availability of resources. What should be done centrally and locally? • The preparation of systematic reviews and evidence profiles should be coordinated centrally, in collaboration with organizations that produce systematic reviews. Centrally developed evidence profiles should be adaptable to specific local circumstances. • Consideration should be given to models that involve central coordination with work being undertaken by centres located throughout the world. • While needs, availability of resources, costs, the presence of modifying factors and values need to be assessed locally, support for undertaking these assessments may be needed to make guidelines applicable. • WHO should provide local support for adapting and implementing recommendations by developing tools, building capacity, learning from international experience, and through international networks that support evidence-informed health policies, such as the Evidence-informed Policy Network (EVIPNet). How should recommendations be adapted? • WHO should provide detailed guidance for adaptation of international recommendations. • Local adaptation processes should be systematic and transparent, they should involve stakeholders, and they should report the key factors that influence decisions, including those flagged in international guidelines, and the reasons for any modifications that are made

Improving the use of research evidence in guideline development: 1. Guidelines for guidelines

BACKGROUND: The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the first of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this. OBJECTIVES: We reviewed the literature on guidelines for the development of guidelines. METHODS: We searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments. KEY QUESTIONS AND ANSWERS: We found no experimental research that compared different formats of guidelines for guidelines or studies that compared different components of guidelines for guidelines. However, there are many examples, surveys and other observational studies that compared the impact of different guideline development documents on guideline quality. WHAT HAVE OTHER ORGANIZATIONS DONE TO DEVELOP GUIDELINES FOR GUIDELINES FROM WHICH WHO CAN LEARN? • Establish a credible, independent committee that evaluates existing methods for developing guidelines or that updates existing ones. • Obtain feedback and approval from various stakeholders during the development process of guidelines for guidelines. • Develop a detailed source document (manual) that guideline developers can use as reference material. WHAT SHOULD BE THE KEY COMPONENTS OF WHO GUIDELINES FOR GUIDELINES? • Guidelines for guidelines should include information and instructions about the following components: 1) Priority setting; 2) Group composition and consultations; 3) Declaration and avoidance of conflicts of interest; 4) Group processes; 5) Identification of important outcomes; 6) Explicit definition of the questions and eligibility criteria ; 7) Type of study designs for different questions; 8) Identification of evidence; 9) Synthesis and presentation of evidence; 10) Specification and integration of values; 11) Making judgments about desirable and undesirable effects; 12) Taking account of equity; 13) Grading evidence and recommendations; 14) Taking account of costs; 15) Adaptation, applicability, transferability of guidelines; 16) Structure of reports; 17) Methods of peer review; 18) Planned methods of dissemination & implementation; 19) Evaluation of the guidelines. WHAT HAVE OTHER ORGANIZATIONS DONE TO IMPLEMENT GUIDELINES FOR GUIDELINES FROM WHICH WHO CAN LEARN? • Obtain buy-in from regions and country level representatives for guidelines for guidelines before dissemination of a revised version. • Disseminate the guidelines for guidelines widely and make them available (e.g. on the Internet). • Develop examples of guidelines that guideline developers can use as models when applying the guidelines for guidelines. • Ensure training sessions for those responsible for developing guidelines. • Continue to monitor the methodological literature on guideline development

Evidence-informed health policy 3 – Interviews with the directors of organizations that support the use of research evidence

Background: Previous surveys of organizations that support the development of evidence-informed health policies have focused on organizations that produce clinical practice guidelines (CPGs) or undertake health technology assessments (HTAs). Only rarely have surveys focused at least in part on units that directly support the use of research evidence in developing health policy on an international, national, and state or provincial level (i.e., government support units, or GSUs) that are in some way successful or innovative or that support the use of research evidence in low- and middle-income countries (LMICs). Methods: We drew on many people and organizations around the world, including our project reference group, to generate a list of organizations to survey. We modified a questionnaire that had been developed originally by the Appraisal of Guidelines, Research and Evaluation in Europe (AGREE) collaboration and adapted one version of the questionnaire for organizations producing CPGs and HTAs, and another for GSUs. We sent the questionnaire by email to 176 organizations and followed up periodically with non-responders by email and telephone. Results: We received completed questionnaires from 152 (86%) organizations. More than one-half of the organizations (and particularly HTA agencies) reported that examples from other countries were helpful in establishing their organization. A higher proportion of GSUs than CPG-or HTA-producing organizations involved target users in the selection of topics or the services undertaken. Most organizations have few (five or fewer) full-time equivalent (FTE) staff. More than four-fifths of organizations reported providing panels with or using systematic reviews. GSUs tended to use a wide variety of explicit valuation processes for the research evidence, but none with the frequency that organizations producing CPGs, HTAs, or both prioritized evidence by its quality. Between one-half and two-thirds of organizations do not collect data systematically about uptake, and roughly the same proportions do not systematically evaluate their usefulness or impact in other ways. Conclusion: The findings from our survey, the most broadly based of its kind, both extend or clarify the applicability of the messages arising from previous surveys and related documentary analyses, such as how the 'principles of evidence-based medicine dominate current guideline programs' and the importance of collaborating with other organizations. The survey also provides a description of the history, structure, processes, outputs, and perceived strengths and weaknesses of existing organizations from which those establishing or leading similar organizations can draw

Incorporating a gender perspective into the development of clinical guidelines: a training course for guideline developers

<p>Abstract</p> <p>Background</p> <p>Dutch guideline-developing organizations do not focus systematically on differences between men and women when developing guidelines, even though there is increasing evidence that being male or female may have an effect on health and health outcomes. In collaboration with two prominent Dutch guideline-developing organizations, we designed a training course to encourage systematic attention to sex differences in guideline development procedures.</p> <p>Methods</p> <p>The course is targeted towards guideline developers. Its aims are to improve awareness concerning the relevance of considering sex differences in the guideline development process, as well as the competence and skills necessary for putting this into practice. The design and teaching methods of the course are based on adult learning styles and principles of changing provider behaviour. It was adjusted to the working methods of guideline organizations. The course was taught to, and evaluated by, a group of staff members from two guideline organizations in the Netherlands.</p> <p>Results</p> <p>The course consists of five modules, each of which corresponds to a key step in the guideline development process. The participants rated the training course positively on content, programme, and trainers. Their written comments suggest that the course met its objectives.</p> <p>Conclusion</p> <p>The training course is the first to address sex differences in guideline development. Results from the pilot test suggest that the course achieved its objectives. Because its modules and teaching methods of the course are widely transferable, the course could be useful for many organizations that are involved in developing guidelines. Follow-up studies are needed to assess the long-term effect of the course on the actions of guideline developers and its utility in other settings.</p